Papers with language processing tasks
Do Diacritics Matter? Evaluating the Impact of Arabic Diacritics on Tokenization and LLM Benchmarks (2026.findings-eacl)
Copied to clipboard
| Challenge: | Diacritics can significantly influence language processing tasks in Arabic . their presence can increase subword fragmentation during tokenization, reducing performance . |
| Approach: | They analyze the impact of diacritics on tokenization and benchmark task performance across major Large Language Models. |
| Outcome: | The proposed model is robust to diacritics, but full diacritization leads to token fragmentation and degraded performance. |
CRAFT Shared Tasks 2019 Overview — Integrated Structure, Semantics, and Coreference (D19-57)
Copied to clipboard
William Baumgartner, Michael Bada, Sampo Pyysalo, Manuel R. Ciosici, Negacy Hailu, Harrison Pielke-Lombardo, Michael Regan, Lawrence Hunter
| Challenge: | CRAFT corpus provides a unique foundation for integrating natural language processing (NLP) tasks involving structure, semantics, and coreference. |
| Approach: | They propose to use the CRAFT corpus to evaluate three fundamental language processing tasks over full-text biomedical articles. |
| Outcome: | The CRAFT corpus provides a unique foundation for integrating natural language processing tasks involving structure, semantics, and coreference. |
Shortcutted Commonsense: Data Spuriousness in Deep Learning of Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has found that commonsense reasoning models are learning transferable generalizations . commonsensibility is a human capacity that has been a core challenge to Artificial Intelligence since its inception. |
| Approach: | They conduct an analysis of benchmarks that involve commonsense reasoning . they find that most datasets experimented with are problematic . commonsensence is a quintessential human capacity . |
| Outcome: | The proposed model is able to perform well on commonsense reasoning tasks . the model is not learning transferable generalizations or taking advantage of shortcuts . |
Biased TextRank: Unsupervised Graph-Based Content Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | TextRank ranks text spans according to their importance for language processing tasks and their relevance to an input “focus.” |
| Approach: | They propose a graph-based content extraction method inspired by TextRank that ranks text spans according to their importance for language processing tasks and according to relevance to an input “focus.” |
| Outcome: | The proposed method improves on two different datasets by significant ROUGE-N score margins. |
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Sentence embeddings are useful for language processing tasks, but it is unclear how to produce them from encoder-decoder models. |
| Approach: | They investigate the effects of scaling up sentence encoders to 11B parameters on sentence embeddings from text-to-text transformers (T5) . |
| Outcome: | The proposed models outperform the previous best models on both SentEval and SentGLUE transfer tasks. |
Multimodal Grounding for Language Processing (C18-1)
Copied to clipboard
| Challenge: | Recent developments in multimodal processing facilitate conceptual grounding of language. |
| Approach: | They analyze multimodal processing to examine the benefits and challenges of multimodal grounding . they focus on multimodal linguistic grounding of verbs which play a crucial role in compositional power of language. |
| Outcome: | The proposed methods improve the cognitive models of human information processing and address the challenges that arise. |
Korean Language Modeling via Syntactic Guide (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing research on pre-trained language models focuses on widely-used languages . however, not every language can benefit from such models due to computational resources . |
| Approach: | They propose to build a pre-trained language model that understands the linguistic phenomena in the target language with low resources. |
| Outcome: | The proposed model improves the performance of Korean language understanding tasks. |
A Chinese Dataset with Negative Full Forms for General Abbreviation Prediction (L18-1)
Copied to clipboard
| Challenge: | a common phenomenon across languages is abbreviation, but it's not always possible to predict it accurately. |
| Approach: | They build a dataset for general Chinese abbreviation prediction using a negative full form . they find that abbrevation prediction can improve the performance of abbreviation recognition . |
| Outcome: | The proposed dataset evaluates models on abbreviation prediction in Chinese . it shows that abbrevation prediction improves performance in language processing tasks . |
Compositional Demographic Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are usually derived from corpora containing text from many individuals . however, they cannot account for user-specific word preferences, such as using the same word in different ways across contexts. |
| Approach: | They propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user. |
| Outcome: | The proposed representations outperform generic representations on two English language tasks. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
Building the Old Javanese Wordnet (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing wordnets for Old Javanese are limited and lack of an open-source version of the language is a barrier to its development. |
| Approach: | They propose to build a machine readable resource for Old Javanese using the Princeton Wordnet's synsets and semantic hierarchy. |
| Outcome: | The wordnet contains 2,054 concepts or synsets and 5,911 senses. |
On the Generalization of Training-based ChatGPT Detection Methods (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training-based methods are ineffective to detect LLM generated texts from unseen tasks or topics which are not collected during training. |
| Approach: | They propose to train classification models to distinguish LLMs from human texts by a distribution shift caused by prompts, text lengths, topics, and language tasks. |
| Outcome: | The proposed methods can detect LLMs from black-box models, but they suffer from distribution shifts due to a wide range of factors, including prompts, text lengths, topics, and language tasks. |
Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey (2025.findings-naacl)
Copied to clipboard
Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, Julian McAuley, Wei Ai, Furong Huang
| Challenge: | Large Language Models (LLMs) have shown great potential to enhance Natural Language Processing (NLP) models in areas such as predictive accuracy, fairness, robustness, and explainability. |
| Approach: | They evaluate or improve generative Large Language Models from a causal perspective in areas such as reasoning capacity, fairness and safety issues, explainability, and handling multimodality. |
| Outcome: | The proposed models can be used to perform causal relationship discovery and causal effect estimation tasks. |
Unifying Data Perspectivism and Personalization: An Application to Social Norms (2022.emnlp-main)
Copied to clipboard
| Challenge: | Obtaining a single ground truth is not possible or necessary for subjective tasks. |
| Approach: | They propose a set of personalization methods to model annotators and compare their effectiveness for predicting social norms. |
| Outcome: | The proposed model outperforms existing models and compares performance across subsets of social situations that vary by the closeness of the relationship between parties in conflict. |
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)
Copied to clipboard
Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng
| Challenge: | Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging. |
| Approach: | They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model. |
| Outcome: | The proposed method is comparable to existing methods and comparable to those using historical data. |
Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current fine-tuning methods to adapt LLMs for simultaneous translation suffer from several issues, such as unnecessarily expanded training sets, increased prompt sizes, or restriction to a single decision policy. |
| Approach: | They propose a new paradigm for fine-tuning large language models for simultaneous translation using an attention mask approach. |
| Outcome: | The proposed model improves translation quality compared to state-of-the-art models on five language pairs while reducing the computational cost. |